Skip to content

branch-4.1: [fix](test) bound recycler polling waits - #66237

Draft
hello-stephen wants to merge 2 commits into
apache:branch-4.1from
hello-stephen:agent/recycler-check-timeout
Draft

branch-4.1: [fix](test) bound recycler polling waits#66237
hello-stephen wants to merge 2 commits into
apache:branch-4.1from
hello-stephen:agent/recycler-check-timeout

Conversation

@hello-stephen

@hello-stephen hello-stephen commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

What changed

Add a 30-minute deadline to every unbounded Recycler/Checker polling loop under cloud_p0/recycler.

The change now covers all four unbounded loops across the three related cases:

  • test_recycler.groovy: wait for a Recycler round and wait for a Checker round
  • check_meta.groovy: wait for a successful Recycler round
  • test_checker.groovy: wait for the expected Checker failure

All other Recycler cases already use bounded retry counters and are unchanged. When a service timestamp does not advance, the affected test now fails with the relevant timestamps instead of looping until TeamCity’s 24-hour build timeout.

Why

The branch-4.1 Recycler pipeline has repeatedly remained inside an unbounded polling loop with no test occurrence until TeamCity terminated the build after 1440 minutes (for example, builds 204533 and 204268). A normal successful test_recycler execution in build 203828 took about 11 minutes, so a 30-minute per-phase deadline keeps reasonable headroom while turning an infrastructure/service stall into an actionable failure.

This does not suppress a Recycler or Checker failure and does not change any success condition. It only bounds waits and preserves the last observed job timestamps in the error message.

Validation

  • git diff --check
  • Scanned regression-test/suites/cloud_p0/recycler: 4 unbounded while (true) loops across 3 files, all 4 now have timeout guards
  • All other polling loops in the Recycler directory already have explicit retry bounds
  • Compared with successful test_recycler duration in branch-4.1 build 203828
  • Current build 204938 has remained in the same unbounded loop for hours with no test occurrence, reproducing the need for this guard

Related: DORIS-27416

@hello-stephen

Copy link
Copy Markdown
Contributor Author

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hello-stephen

Copy link
Copy Markdown
Contributor Author

[branch41-macos13-runner-retirement]

The BE UT (macOS) job is blocked by a shared workflow issue, not this PR:

  • job 90582616675 has been queued since 2026-07-29 12:58 UTC
  • it requests the retired macos-13 image and has no runner assignment
  • GitHub retired that label on December 4, 2025

Draft PR #66248 switches branch-4.1 to the supported Intel label macos-15-intel, preserving the existing x86_64 third-party package and JAVA_HOME_17_X64. Its own macOS job 90720706740 was assigned a GitHub-hosted Intel runner immediately and is currently in progress.

After #66248 merges, this PR's macOS check still needs to be retriggered and reach a terminal result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant